Tight Sample Complexity of Large-Margin Learning
نویسندگان
چکیده
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution’s covariance matrix, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the γ-adapted-dimension of the source distribution. We conclude that this new quantity tightly characterizes the true sample complexity of large-margin classification. The bounds hold for a rich family of sub-Gaussian distributions.
منابع مشابه
Distribution-dependent sample complexity of large margin learning
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the margin-adapted dimension of the ...
متن کاملOn the Complexity of Good Samples for Learning ?
In machine-learning, maximizing the sample margin can reduce the learning generalization-error. Thus samples on which the target function has a large margin (γ) convey more information so we expect fewer such samples. In this paper, we estimate the complexity of a class of sets of large-margin samples for a general learning problem over a finite domain. We obtain an explicit dependence of this ...
متن کاملImproved Guarantees for Learning via Similarity Functions
We continue the investigation of natural conditions for a similarity function to allow learning, without requiring the similarity function to be a valid kernel, or referring to an implicit high-dimensional space. We provide a new notion of a “good similarity function” that builds upon the previous definition of Balcan and Blum (2006) but improves on it in two important ways. First, as with the ...
متن کاملLarge Margin Boltzmann Machines
Boltzmann Machines are a powerful class of undirected graphical models. Originally proposed as artificial neural networks, they can be regarded as a type of Markov Random Field in which the connection weights between nodes are symmetric and learned from data. They are also closely related to recent models such as Markov logic networks and Conditional RandomFields. Amajor challenge for Boltzmann...
متن کاملThe Rademacher Complexity of Linear Transformation Classes
Bounds are given for the empirical and expected Rademacher complexity of classes of linear transformations from a Hilbert space H to a nite dimensional space. The results imply generalization guarantees for graph regularization and multi-task subspace learning. 1 Introduction Rademacher averages have been introduced to learning theory as an e¢ cient complexity measure for function classes, mot...
متن کامل